"The highlighted tokens are often function words, affixes, or short morphemes in various languages, including prepositions, conjunctions, pronouns, and grammatical particles, as well as some high-activation non-standard or corrupted characters. These tokens are typically important for sentence structure, meaning, or language identification."
Score Type | Accuracy | Precision | Recall | F1 score | TPR | TNR | FPR | FNR |
---|---|---|---|---|---|---|---|---|
detection | 0.52 | 0.51 | 1.0 | 0.676 | 1.0 | 0.04 | 0.96 | 0.0 |
fuzz | 0.47 | 0.484 | 0.9 | 0.629 | 0.9 | 0.04 | 0.96 | 0.1 |